HW2 Submission¶

Damanpreet Singh¶

Andrew ID : damanprs¶

q1¶

(a) Stanford Bunny¶

For this part:

  • I loaded up the given 2D-3D correspondences, split them up and projected and normalized them
  • The points are then stacked up to form a lineaar equation of $Ap =0$, where $p$ is the flattened camera mtrix $P$

  • The linear equation is derived from the following image.png

  • $A$ matrix is the $3\times12$ matrix shown, formed by stacking up 2D-3D correspondences

  • After getting the $A$ matrix, I get the $p$ matrix by taking an SVD of the $A$ matrix
  • $P$ matrix is extracted by re-shaping the $p$ vector
  • The camera matrix is then used to transform the given surface points into the image plane using $x = PX$
  • Similarly, for the bounding box, the outer points are projected, normalized and then transformed to the image plane
  • The camera matrix came out as $\begin{bmatrix} 6.50945644e+03 &-2.93252813e+03& 1.06713749e+03 & 2.22753498e+03 \\-8.60882948e+02& -6.73715135e+03 & 1.96129030e+03 &1.82145602e+03\\ 6.22451352e-01& -1.41369464e+00 &-7.89678657e-01& 1.00000000e+00\end{bmatrix}$
  • The results are shown below
Input Image
bunny.jpeg
Surface Points Bounding Box
bunny_pts.jpg bunny_bbox.jpg

(b) Cuboid¶

For this part

  • I clicked a picture of a cuboid, shown below
  • Took measurements of the box and added them as 3D points, for correspondence. The points are as shown

$\begin{bmatrix} 0&0&0\\ 8.75&0&0\\0&12.25&0\\0&0&5.25\\8.75&0&5.25\\0&12.25&5.25 \end{bmatrix}$

  • Then, annotations are drawn on the captured image, to define 2D-3D correspondences
  • Camera matrix is calculated using the same code as in last one
  • I drew boundaries on the front 2 faces of the cuboid
  • To do this, I gave the coordinates of the 3D points for the lines and transformed them using the $P$ matrix into image plane and drew lines usng these transformed co-ordinates
  • The camera matrix came out as $\begin{bmatrix} 6.30931466e+01 &-2.43307628e+01 &-1.05598007e+00 &4.46995307e+02 \\ 1.85849948e+01& 1.39298223e+01& -6.35452400e+01 &8.49999815e+02 \\ 4.31643779e-02 & 3.98216103e-02 &-7.31243463e-03 &1.00000000e+00 \end{bmatrix}$
  • The results are shown below
Input Image Annotated 2D points Result
cuboid.jpg cuboid_annotation.jpg cuboid_lines.jpg

q2¶

(a)¶

For this part

  • I loaded up the given points for the set of parallel lines
  • I made lines, by projecting and normalizing these points
  • Using these $\mathbb{P}^2$ lines, I calculated the vanishing points, using $v = l_1 \times l_2$, where $l_1$ and $l_2$ are parallel lines
  • These vanishing points are then used to get the Image of Absolute Conic($\omega$), using $v_1^T \omega v_2 =0$
  • $v_1^T\omega v_2$ yields a linear equation as

$\begin{bmatrix} x_1y_1+x_2y_2& x_1y_3+x_3y_1 & x_2y_3+x_3y_2 & x_3y_3 \end{bmatrix} \begin{bmatrix} \omega_1 \\ \omega_2 \\ \omega_3 \\ \omega_4 \end{bmatrix} =0 $

  • Here, $v_1 = \begin{bmatrix} x_1 &x_2 &x_3 \end{bmatrix}^T$, $v_2 = \begin{bmatrix} y_1 &y_2 &y_3 \end{bmatrix}^T$ and $\omega_{mat} = \begin{bmatrix} \omega_1 &0&\omega_2 \\ 0&\omega_1 & \omega_3 \\ \omega_2 & \omega_3 & \omega_4 \end{bmatrix}$
  • $\omega_{mat}$ matrix has such structure due to assumptions of zero skew and square pixels
  • Stacking all these combinations of vanishing points up, gives an $A$ matrix, for solving $A \omega =0$
  • $\omega$ is retrieved by taking SVD of $A$
  • The intrinsics are then computed by a cholesky decomposition of the $\omega_{mat}$ matrix, as $\omega_{mat} = K^{-T}K^{-1}$
  • The K matrix camee out to be

$\begin{bmatrix} 1.15417802e+03& 0.00000000e+00 &5.75066005e+02\\ 0.00000000e+00 &1.15417802e+03 &4.31939090e+02\\ 0.00000000e+00 &0.00000000e+00 &1.00000000e+00 \end{bmatrix}$

  • The results are shown below
Input Image Annotated Parallel Lines Vanishing Points
q2a.png q2a_parallel_annotated.png building_vanish_pts.jpg

(b)¶

For this part

  • The provided points for sqaures were loaded up, and projected and normailized
  • Points for each sqaure, were then evaluated for homography from the metric space
  • Sqaure in metric projective space was assumed, with the corner points shown below $\begin{bmatrix}0&1&1\\1&1&1\\1&0&1\\0&0&1 \end{bmatrix}$
  • Homography was estimated, using the algorithm used in last assignment

  • Once homographies from all the squares are recieved, then they are used to calculate the IAC, using the given relations

    image.png

    image.png

  • Assuming $h_1 = \begin{bmatrix} x_1& x_2&x_3\end{bmatrix}^T$, $h_2 = \begin{bmatrix} y_1&y_2&y_3\end{bmatrix}^T$, $\omega = \begin{bmatrix} \omega_1&\omega_2&\omega_3 \\ \omega_2&\omega_4 & \omega_5 \\ \omega_3 & \omega_5 & \omega_6 \end{bmatrix}$, each homography yields two eqautions for a linear equation system of $Ax=0$, where $x=\begin{bmatrix} \omega_1 & \omega_2 & \omega_3 &\omega_4 & \omega_5 & \omega_6 \end{bmatrix}$

  • Here $A = \begin{bmatrix} x_1y_1& x_1y_2+x_2y_1& x_1y_3+x_3y_1&x_2y_2&x_2y_3+x_3y_2&x_3y_3\\ x_1x_1& x_1x_2+x_2x_1& x_1x_3+x_3x_1&x_2x_2&x_2x_3+x_3x_2&x_3x_3\end{bmatrix} - \begin{bmatrix} 0& 0& 0&0&0&0\\ y_1y_1& y_1y_2+y_2y_1& y_1y_3+y_3y_1&y_2y_2&y_2y_3+y_3y_2&y_3y_3\end{bmatrix}$

  • Stacking these six equations, and taking SVD of $A$ yields the $\omega$ matrix

  • The intrinsics can then be derived using a choledky decomposition, as shown in the last part
  • The intrinsic matrix came out to be

$\begin{bmatrix} 1.07692614e+03 &-4.52638065e+00 &5.11568923e+02\\ 0.00000000e+00 & 1.07626752e+03 &3.95526278e+02\\ 0.00000000e+00 &0.00000000e+00 &1.00000000e+00 \end{bmatrix}$

  • The angles for the planes are | | Angles between planes (degrees) | | --- | --- | | Plane 1 & 2 | 67.28242526443618 | | Plane 2 & 3 | 94.71069301662723 | | Plane 3 & 1 | 92.20348558863756 |
  • The annotations are shown below | Input Image | Annotated Square 1 | Annotated Square 2 | Annotated Square 3 | | --- | --- | --- | --- | | q2b.png | image.png| image.png | image.png |

(c)¶

For this part

  • A custom image was took, which can be seen below
  • The algorithm is completely similar to the previous case, except for the metric space polygon co-ordinates, whihc now are $\begin{bmatrix} 0&1&1\\wh\_ratio&1&1\\wh\_ratio&0&1\\0&0&1 \end{bmatrix}$
  • Here wh_ratio is the width-to-height ratio, which can be determined for each recatngle in the image

  • For our case, the dimensions are | Plane | Width (inches) | Height (inches)| wh_ratio | | --- | --- | --- | --- | | 1 - green| 32.5 | 14.5 | 2.24| | 2 - red| 14|9|1.55| |3 - blue|14 | 9.25 |1.51|

  • The algorithm was completely similar to the last part after this

  • The intrinsics obtained are

$\begin{bmatrix} 953.11573129 &3.95594949 &563.09329766\

  1. &953.16037085 &423.39295839\
  2. &0. &1. \end{bmatrix}$
  • The images used are
Input Image Annotated Squares
rectangle.jpg rectangle_lines.jpg

q3¶

(a)¶

For this part

  • The given plane points were imported
  • Intrinsics matrix was computed using the method given in q2(a). It came out as $\begin{bmatrix} 886.72689287 &0. &606.02323158\
    1. &886.72689287 &450.5126032 \
      1. &0. &1. \end{bmatrix}$
  • The normals for each point is calculated using the given points. Using the plane points for a plane, the vanishing points $v_1, v_2$ can be calculated. Then, the 3D directino vector for these points is calculated as $d = K^{-1}v$
  • The normal for any plane comes out to be $n = d_1\times d_2$, where $d_1$ and $d_2$ are the 3D direction vectors for the vanishing points.
  • The equation for any plane now, is $n^TX +a=0$, in euclidean co-ordinates
  • The point at the intersection of the grass plane and the two building planes is chosen as the main reference point, marked as 2,8,10 in the image below. The scale for this is chosen as 1
  • The 3D co-ordincate corresponding to this $x_{ref}$ is found as $X_{ref} = \lambda d_{ref},\hspace{0.2cm} d_{ref} = K^{-1}x_{ref}$. Here $\lambda =1$, so $X_{ref} = K^{-1}x_{ref}$
  • The plane numbering used |Plane|Boundary Color| | --- | --- | | 1 | Green| | 2 | Yellow| | 3 | Cyan| | 4 | Blue| | 5 | Red|
  • This point lies on first three planes. So, the constant for the plane equations for these planes can be calcualted as $a = -n^T X_{ref}$, where $n$ is the normal to that plane
  • $a_1,a_2,a_3$ have been calculated. To calculate the constants for the upper 2 planes (plane #4,5), a second refernece point is chosen, the one at the intersection of plane 1,2,4,5, marked as 1,4,15,18 in the annotations below, $x_{ref2}$
  • We know $n_1,a_1$ and that $x_{ref2}$ lies on plane 1. As $X_{ref2} = \lambda_{ref2}d_{ref2} , \hspace{0,2cm} d_{ref2} = (K^{-1}x_{ref2})$, we can calculate $\lambda_{ref2}$ as

$\lambda_{ref2} = \frac{-a_1}{-n_1^Td_{ref2}} $

  • Using this second refernce point values, we can get all parameters for plane #4,5
  • Now, the points lying inside each annotated plane is found using the python package shapely. "shapely.geometry.Polygon"
  • These points are then transformed to 3D using $X = \lambda d , \hspace{0,2cm} d = (K^{-1}x)$, we can calculate $\lambda$ as

$\lambda = \frac{-a}{-n^Td} $

  • Here $a$ and $n$ is of the plane the point belongs to.
  • Then, once 3D co-ordinates for all poiunts are known, they are plotted using "matplotlib scatter3D"
  • The image, annotations and reconstructions can be seen below
Input Image Annotations
q3.png image.png
  • Reconstruction 1

3a_recon_1.png

  • Reconstruction 2

3a_recon_2.png

  • Reconstruction 3

3a_recon_3.png

Late Days¶

image.png